Identifying a change in a home environment

ABSTRACT

A system and computer implemented method for detecting a change in a home environment for a visually impaired user is disclosed. The method may include monitoring the home environment using a set of sensors configured to collect environment data for a set of areas of the home environment. The method may also include detecting an environmental change in an area of the home environment. The method may also include providing, in response to a triggering event, a notification of the environmental change.

FIELD

The present disclosure relates to computer-aided navigation, and morespecifically, identifying a change in a home environment for a visuallyimpaired user.

BACKGROUND

Various methods and systems can aid visually impaired individuals inunfamiliar surroundings. Devices and systems may guide the visuallyimpaired by utilizing detection devices to warn the user of obstacles.Such devices may allow for the user to navigate a home environment.

SUMMARY

Aspects of the present disclosure, in certain embodiments, are directedtoward detecting a change in a home environment for a visually impaireduser. In certain embodiments, the method may include monitoring the homeenvironment using a set of sensors configured to collect environmentdata for a set of areas of the home environment. In certain embodiments,the method may include detecting an environmental change in an area ofthe home environment. In certain embodiments, the method may includeproviding, in response to a triggering event, a notification of theenvironmental change.

The above summary is not intended to describe each illustratedembodiment or every implementation of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings included in the present application are incorporated into,and form part of, the specification. They illustrate embodiments of thepresent disclosure and, along with the description, serve to explain theprinciples of the disclosure. The drawings are only illustrative ofcertain embodiments and do not limit the disclosure.

FIG. 1 illustrates a method for detecting a change in a home environmentfor a visually impaired user, according to embodiments;

FIG. 2 is an illustration of a home environment configured for motionanalysis, according to embodiments;

FIG. 3 is an illustration of a home environment configured for imageanalysis, according to embodiments;

FIG. 4 is an illustration of a home environment configured for soundanalysis, according to embodiments; and

FIG. 5 depicts a high-level block diagram of a computer system forimplementing various embodiments, according to embodiments.

While the invention is amenable to various modifications and alternativeforms, specifics thereof have been shown by way of example in thedrawings and will be described in detail. It should be understood,however, that the intention is not to limit the invention to theparticular embodiments described. On the contrary, the intention is tocover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the invention.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to various embodiments of asystem and method for detecting a change in a home environment for avisually impaired user. More particular aspects relate to using imageprocessing, sonar mapping, and motion sensing systems to detect thechange in the home environment for the visually impaired user. Themethod may include monitoring the home environment using a set ofsensors configured to collect environment data for a set of areas of thehome environment. The method may also include detecting an environmentalchange in an area of the home environment. Additionally, in certainembodiments, the method may include providing a notification of theenvironmental change. The notification of the environmental change maybe provided in response to a triggering event.

For a visually impaired person, navigating from one location to anothercan pose a variety of challenges. If a visually impaired person is notfamiliar with an environment, there may be a number of obstacles thatthe he or she carefully maneuvers to arrive at his or her intendeddestination. When a visually impaired person is navigating a knownenvironment (e.g., a home environment or location they are familiarwith), he or she may memorize the location of objects and the number ofsteps from one location to another. However, aspects of the presentdisclosure relate to the recognition that, in certain situations, anunexpected obstacle in a home environment (e.g., a piece of furniture orother object that has been moved within the home) may be potentiallyhazardous to a visually impaired person, as he or she may not be ontheir guard for obstacles when traveling in a familiar environment.Accordingly, aspects of the present disclosure, in certain embodiments,relate to a system and method for monitoring a home environment usingimage processing, sonar mapping, and motion sensing systems, anddetecting a potential obstacle. A notification of the potential obstaclemay be provided to the user of the system. Aspects of the presentdisclosure may provide benefits associated with confidence, efficiency,and safety (e.g., faster and safer home environment navigation) for avisually impaired user.

Aspects of the present disclosure relate to various embodiments of asystem and method for detecting a change in a home environment for avisually impaired user. More particular aspects relate to using imageprocessing, sonar mapping, and motion sensing systems to detect thechange in the home environment for the visually impaired user. Themethod and system may work on a number of devices and operating systems.Aspects of the present disclosure include monitoring a home environmentusing a set of sensors configured to collect environment data for a setof areas of the home environment. The set of sensors can include motionsensors, audio sensors, cameras, or other types of sensors. The methodmay also include detecting an environmental change in an area of thehome environment. The environmental change may be a movement of anobject (e.g., an object that may be an obstacle for a visually impaireduser), a broken object, a substantial change in temperature, or anotherchange that may affect the visually impaired user. The method may alsoinclude providing, in response to a triggering event, a notification ofthe environmental change.

In certain embodiments, collecting the environment data may includecapturing, by an image capturing system, a first set of images for theset of areas of the home environment. In certain embodiments, detectingthe environmental change in the area of the home environment includescapturing, in response to receiving a motion alert from a motiondetection system coupled to the image capturing system, a second set ofimages for the set of areas. The method may then include identifying anobject of the second set of images meeting an obstruction criteria. Theobstruction criteria may include an object displacement threshold basedon an origin position and a subsequent position, and an object sizethreshold based on a first dimension, a second dimension, and a thirddimension. In certain embodiments, providing the notification of theenvironmental change may include notifying a user, using a mobile devicecommunicatively connected to the image capturing system, of theidentified object meeting the obstruction criteria in response todetecting a home arrival. In certain embodiments, the home arrival maybe detected by a security system in response to a door unlock of a home.In certain embodiments, detecting the home arrival may includerecording, by a sound capture device, a set of dog barks (including likenoises, such as panting). In response to recording the set of dog barks,the method may include analyzing a bark frequency and a bark pitch ofthe set of dog barks. The method may include determining, based on thebark frequency and the bark pitch, occurrence of the home arrival.

In certain embodiments, collecting the environment data may includemapping, using a sonar mapping system, a first sonar map of the set ofareas of the home environment. In certain embodiments, detecting theenvironmental change in the area of the home environment may includemapping, in response to receiving a noise alert from a sound detectionsystem coupled to the sonar mapping system, a second sonar map of theset of areas. The method may also include identifying an object of thesecond sonar map meeting an obstruction criteria. In certainembodiments, providing the notification of the environmental change mayinclude notifying, in response to an alert from a sleep sensor, using amobile device communicatively connected to the image capturing system, auser of the identified object meeting the obstruction criteria. Thealert from the sleep sensor may include a change in a pulse beyond apulse threshold, and a change in a respiration frequency beyond arespiration threshold. In certain embodiments, the method may includecollecting, by the sonar mapping system, a sound profile data includinga background noise level of the set of areas. The method may alsoinclude creating, using sound profile data other than the backgroundnoise level, the sonar map.

In certain embodiments, collecting the environment data may includecapturing, by an image capturing system, a first set of images for theset of areas of the home environment. In certain embodiments, detectingthe environmental change in the area of the home environment includescapturing, in response to receiving a motion alert from a motiondetection system coupled to the image capturing system, a second set ofimages for the set of areas. The method may then include identifying anobject of the second set of images meeting an obstruction criteria. Theobstruction criteria may include an object displacement threshold basedon an origin position and a subsequent position, and an object sizethreshold based on a first dimension, a second dimension, and a thirddimension. In certain embodiments, providing the notification of theenvironmental change may include notifying a user, using a mobile devicecommunicatively connected to the image capturing system, of theidentified object meeting the obstruction criteria in response todetecting an area transition. In certain embodiments, detecting the areatransition may include monitoring the location of a cane of the user.The cane may be equipped with a RFID (radio-frequency identification)bit communicatively connected to a RFID tracking system. The method mayalso include detecting that the location of the cane has been displacedby a distance value greater than a distance threshold. In certainembodiments, detecting the area transition may include using a thermalimaging system to monitor a body heat signature. The method may alsoinclude detecting that the body heat signature has been displaced by adistance value greater than a distance threshold.

Turning now to the figures, FIG. 1 illustrates a method 100 fordetecting a change in a home environment for a visually impaired user,consistent with various embodiments of the present disclosure. Aspectsof FIG. 1 are directed toward using image capturing, sound capturing,and motion sensing systems to detect the change in the home environmentfor the visually impaired user. The method 100 may begin at block 102and end at block 124. Consistent with various embodiments, the method100 can include a monitoring block 104, an image capturing block 104, asound capturing block 106, a motion sensing block 108, a detecting block110, an image processing block 112, a sonar mapping block 114, aproviding block 116, a home arrival block 118, a sleep sensor block 120,and an area transition block 122.

Consistent with various embodiments of the present disclosure, at block104 the method 100 can include monitoring a home environment using a setof sensors configured to collect environment data for a set of areas ofthe home environment. The home environment may be a shelter or residenceof a person, a family, or a household. In certain embodiments, the homeenvironment may, for example, be a house, apartment, townhome,condominium, or other residence. The home environment may be equippedwith a set of sensors configured to monitor the home environment. Theset of sensors may include motion sensors, audio sensors, cameras, orother types of sensors that can collect data about the surroundingenvironment. The set of sensors may, in certain embodiments, be one ormore household electronic devices stationed within the home that servemultiple functions. For example, the sensors may include webcams,televisions equipped to receive voice commands, phones, intercomsystems, security cameras and microphones, stereos, and the like. Incertain embodiments, the set of sensors may include specializedequipment including cameras, microphones, motion sensors, thermalimaging systems, RFID (radio-frequency identification) tracking systems,or other types of sensors.

In certain embodiments, the set of sensors may be equipped to collectenvironment data for a set of areas of the home environment. Theenvironment data may be video, audio, thermal, motion, infrared, orother data containing information related to the home environment. Forexample, the environment data may be video footage or still images,captured sounds, heat signatures, or other data collected by the set ofsensors. As shown in FIG. 1, in certain embodiments, monitoring the homeenvironment can include image capturing 104, sound capturing 106, andmotion sensing 108. At image capturing block 104, method 100 can includeusing an image capturing system (e.g., cameras, etc.) to capture a setof images of one or more areas of the home environment. At soundcapturing block 106, the method 100 can include using a sound capturingsystem to collect sound and noise data for one or more areas of the homeenvironment. At motion sensing block 108, the method 100 can includeusing a motion-sensing system to collect motion data for one or moreareas of the home environment. The areas of the home environment may beone or more regions or subdivisions of the home environment. Thesubdivisions of the home environment may be based on physicalconstructions (e.g., walls, doors, etc.) or based on function (kitchen,dining room, living room, etc.). As an example, the areas of the homeenvironment may include a bedroom, a kitchen, a living room, a bathroom,and the like.

Consistent with various embodiments, at block 110 the method 100 caninclude detecting an environmental change in an area of the homeenvironment. The change in the home environment can be an alteration ofthe state of one or more aspects of the home environment. For example,the environmental change may be a temperature change beyond a thresholdlevel, an object that has been added to or removed from an area of thehome environment, a malfunction of an appliance, an unidentified personor animal, a noise, and the like. The change in the home environment canbe detected in one of a number of ways. As shown in FIG. 1, detectingthe change in the home environment can include image processing 112 andsonar mapping 114. At image processing block 112, the method 100 caninclude using image processing techniques (e.g., digital image analysisalgorithms, object recognition techniques, etc.) to detect the change inthe home environment. As an example, in certain embodiments, imageprocessing techniques may be used to compare a first set of images of anarea of the home environment to a second set of images of the area ofthe home environment. Accordingly, the method 100 can include detectingan environmental change such as an object from the second set of imagesthat was not present in the first set of images. At block 114, themethod 100 can include using sonar mapping techniques to create a sonarmap of an area of the home environment. As an example, in certainembodiments, at block 114 the method 100 can include comparing a firstsonar map of an area of the home environment to a second sonar map of anarea of the home environment, and identifying an object in the secondsonar map that was not present in the first sonar map.

Consistent with various embodiments, at block 116 the method 100 caninclude providing, in response to a triggering event, a notification ofthe environmental change. The notification of the environmental changemay include informing a user of the time, location, and nature of theenvironmental change. Consistent with various embodiments, thenotification of the environmental change may be provided in one of anumber of ways. For example, the notification may be a voice messagethat is announced in one or more areas of the home environment via aspeaker, intercom, stereo, television, radio, or other device. Incertain embodiments, the notification of the environmental change may beprovided via a mobile device of the user in the form of a voice message,text message, vibrate alert, or sound alert. The notification of theenvironmental change may be provided to the user in response to atriggering event. The triggering event may include an occurrence,trigger, or stimuli that can be detected by the set of sensors or othermethod. Consistent with embodiments of the present disclosure, thetriggering event can include a home arrival 118, a sleep sensor 120, oran area transition 122. Other triggering events are also contemplated.In certain embodiments, the home arrival 118 can include detecting thata known user has arrived at the home environment. In certainembodiments, the sleep sensor 120 can be configured to detect when auser has awoken from sleeping, or arisen from bed. In certainembodiments, the area transition 122 can include detecting that a userhas moved from one area (e.g., room) to another area of the homeenvironment.

Aspects of the present disclosure, in certain embodiments, are directedtoward monitoring the home environment in response to the triggeringevent. In certain embodiments, the set of sensors may be in a standbymode, and be configured to detect one of a number of triggering events.In certain embodiments, the triggering event may be a scheduled timetrigger. Put differently, in certain embodiments, the set of sensors maybe configured to monitor the home environment at a set of configurabletime intervals. Consider the following example. The set of sensors maybe configured to monitor the home environment for changes each hour ofthe day (e.g., on each hour). While a user who lives in the homeenvironment is on vacation, another individual may enter the homeenvironment at 11:04 AM to water one or more plants of the homeenvironment. The individual may move one or more objects or pieces offurniture while watering the plants, and replace them in their originalposition before leaving the home environment at 11:49. At 12:00 PM, theset of sensors may monitor for a change in the home environment, and notdetect a change in the home environment.

FIG. 2 is an illustration of a home environment 200 configured formotion analysis, consistent with embodiments of the present disclosure.Aspects of FIG. 2 are directed toward using one or more motion detecting(similarly referred to herein as motion sensing) devices 202, 204, 206to collect environment data about one or more areas and identify achange in the home environment for a visually impaired user. Asdescribed herein, in certain embodiments, the home environment 200 maybe a house, apartment, townhome, condominium, or other residence. Thehome environment 200 may include one or more areas, including a masterbedroom, additional bedrooms, a family room, dining room, living room,kitchen, and multiple bathrooms. Other areas are also possible. As shownin FIG. 2, the motion sensing devices 202, 204, 206 may be locatedthroughout the home environment 200. For example, in certainembodiments, the motion sensing device 202 may be located in the masterbedroom, the motion sensing device 204 may be located in a hallway, andthe motion sensing device 206 may be located in the living room of thehome environment 200. Consistent with embodiments of the presentdisclosure, the motion sensing devices 202, 204, 206 may be householdappliances configured for sound capture, such as televisions, radios,webcams, security systems, stereos, computers, mobile devices, and thelike.

Aspects of the present disclosure, in certain embodiments, are directedtoward a home environment 200 equipped with one or more motion sensingdevices 202, 204, 206 configured to collect environmental data about oneor more areas of the home environment, and identify a change in the homeenvironment for a visually impaired user. The motion sensing devices202, 204, 206 may be one or a combination of multiple differentimplementations. For example, the motion sensing devices 202, 204, 206may include passive and active infrared sensors, optics sensors (e.g.,video and camera systems), radio frequency systems (e.g., radar,microwave, and tomographic motion detection), sound (e.g. microphonesand acoustic sensors), vibration (triboelectric, seismic, andinertia-switch sensors), magnetism-based sensors (e.g., magnetic sensorsand magnetometers) and the like.

Consistent with various embodiments, the motion sensing devices 202,204, 206 may be configured to monitor and record the time and locationof motion that occurs within the home environment 200. Consider thefollowing example. In certain embodiments, a user (e.g., a visuallyimpaired user) may depart the home environment 200 in the morning whenhe or she leaves for work. While the user is out, the breeze through anopen window may tip over a coat rack nearby the entrance to the homeenvironment, causing it fall in front of the entrance. Accordingly, oneor more of the motion sensing devices 202, 204, 206 may detect that thecoat rack has fallen. In certain embodiments (described herein) the usermay be notified that the coat rack has fallen, and be careful of thecoat rack when he or she returns home. Although three motion sensingdevices are depicted herein, greater or lesser numbers of motion sensingdevices are also possible.

In certain embodiments, the motion sensing devices 202, 204, 206 may beconfigured to only monitor for movement during certain times of the day.For example, the motion sensing devices 202, 204, 206 may be equipped tomonitor the home environment between the hours of 9 AM and 5 PM, while avisually impaired user is at work. In certain embodiments, the motionsensing devices 202, 204, 206 may be configured to monitor particularsubsections of an area, such as in front of a door or nearby a table. Incertain embodiments, the motion sensing devices 202, 204, 206 may beconfigured to monitor above or below a particular height threshold. Forexample, in certain embodiments, the motion sensing devices 202, 204,206 may be configured to monitor for motion above a height of 1 meterfrom the floor. Such a configuration may, for instance, be useful forhome environments with pets or small children.

FIG. 3 is an illustration of a home environment 300 configured for imageanalysis, consistent with embodiments of the present disclosure. Aspectsof FIG. 3 are directed toward using one or more image capture devices302, 304, 306, 308 to collect environment data about one or more areasand identify a change in the home environment for a visually impaireduser. As described herein, in certain embodiments, the home environment300 may be a house, apartment, townhome, condominium, or otherresidence. The home environment 300 may include one or more areas,including a master bedroom, additional bedrooms, a family room, diningroom, living room, kitchen, and multiple bathrooms. Other areas are alsopossible. As shown in FIG. 3, the image capture devices 302, 304, 306,308 may be located throughout the home environment 300. For example, incertain embodiments, the image capture device 302 may be located in themaster bedroom, the image capture device 304 may be located in thefamily room, the image capture device 306 may be located in the diningroom, and the image capture device 308 may be located in the living roomof the home environment 300. Consistent with embodiments of the presentdisclosure, the image capture devices 302, 304, 306, 308 may behousehold appliances configured for sound capture, such as televisions,radios, webcams, security systems, stereos, computers, mobile devices,and the like.

Aspects of the present disclosure, in certain embodiments, are directedtoward a home environment 300 equipped with one or more image capturedevices 302, 304, 306, 308 configured to collect environmental dataabout one or more areas of the home environment, and identify a changein the home environment for a visually impaired user. The image capturedevices 302, 304, 306, 308 may be one or a combination of multipledifferent implementations. For example, the image capture devices 302,304, 306, 308 may include cameras capable of capturing still images aswell as videos of the home environment 300. The images and videocaptured by the image capture devices 302, 304, 306, 308 may bemaintained on local storage devices or on remote storage devices (e.g.,the cloud, etc.) communicatively connected with the image capturedevices 302, 304, 306, 308.

Consistent with various embodiments, the image capture devices 302, 304,306, 308 may be configured to collect environment data including a firstset of images of one or more areas of the home environment 300. Thefirst set of images may serve as a “baseline” of the home environment300, or a reference for use in identifying a change in the homeenvironment 300. As an example, the image capture devices 302, 304, 306,308 may capture a first set of images of the master bedroom, familyroom, dining room, and living room. In certain embodiments, the imagecapture devices 302, 304, 306, 308 may be communicatively connected tothe motion capturing devices 202, 204, 206 described in FIG. 2.Accordingly, in certain embodiments, in response to receiving a motionalert from one or more of the motion capturing devices 202, 204, 206 ofFIG. 2, the image capture devices 302, 304, 306, 308 may be configuredto capture a second set of images for the set of areas of the homeenvironment 300. The motion alert from the motion capturing devices 202,204, 206 may be a notification of motion detected in one or more areasof the home environment 300. The second set of images may be configuredto capture the same areas of the home environment 300 as captured by thefirst set of images.

In certain embodiments, the image capture devices 302, 304, 306, 308 canbe configured to utilize digital image processing techniques to analyzethe first set of images and the second set of images in order toidentify a change (e.g., an object) of the second set of images meetingan obstruction criteria. In certain embodiments, identifying the objectin the second set of images meeting the obstruction criteria may includeusing a digital image processing algorithm. The digital image processingalgorithm can compare the first set of images with the second set ofimages, and use one or more object recognition techniques to identifythe object in the second set of images. A variety of object recognitiontechniques are possible. For example, the digital image processingalgorithm may use edge matching (e.g., Carny edge detection), greyscalematching, gradient matching, feature based methods, geometric hashing,Speeded Up Robust Features (SURF) techniques, and the like. In certainembodiments, the digital image processing algorithm may useScale-Invariant Feature Transform (SIFT) to identify the image.Accordingly, the SIFT technique can use one or more “training images” toextract features descriptions of various objects, and match features ofthe object in the second set of images with the feature descriptions ofthe objects in the training images to identify the object.

Consider the following example. The SIFT technique may use trainingimages of potted plants, packages, chairs, coat racks, desks, and otherobjects to identify feature descriptions such as the dimensions, shape,light reflective properties (e.g., response to illumination changes),and other key points for each object of the training images.Accordingly, in embodiments of the present disclosure, the SIFTtechnique may identify an object by comparing the first set of images tothe second set of images, and determining that an object in the secondset of images was not present in the first set of images, and matchesthe feature description of a known object.

As described herein, in certain embodiments, the object of the secondset of images may meet an obstruction criteria. The digital imageprocessing technique can include determining whether a given object inthe set of images meets the obstruction technique. The obstructioncriteria may include one or more factors that indicate that the objectmay be a potential obstacle or hazard for a user (e.g., a visuallyimpaired user) traversing the area in which the object is located. Incertain embodiments, the obstruction criteria can include a displacementthreshold and an object size threshold. The obstruction criteria may befulfilled if an object meets either the displacement threshold or theobject size threshold. The displacement threshold may be a distance thatan object has moved between an origin position of the first set ofimages and a subsequent position in the second set of images. Forexample, in certain embodiments, the displacement threshold may be 20centimeters. Accordingly, an object in the second set of images that hasmoved less than 20 centimeters from its origin position in the first setof images may not meet the obstruction criteria, while an object in thesecond set of images that has moved more than 20 centimeters from itsoriginal position in the first set of images may meet the obstructioncriteria.

In certain embodiments, the object size threshold may be a set ofdimensions. In certain embodiments, the object size threshold mayinclude a length threshold, width threshold, and height threshold. Anobject that exceeds either the length threshold, width threshold, orheight threshold may meet the obstruction criteria. As an example, incertain embodiments, the object size threshold may be a length of 10centimeters, a width of 10 centimeters, and a height of 30 centimeters.Accordingly, a potted plant that is 8 centimeters long, 8 centimeterswide, and 32 centimeters tall may meet the obstruction criteria.Although an object size threshold and a displacement threshold arespecifically mentioned herein, other obstruction criteria (e.g., atemperature threshold, etc.) are also possible. In certain embodiments,the obstruction criteria can be managed by a mobile device. For example,the mobile device can include a set of preferences that allow a user todefine the displacement threshold and the object size threshold.

Consistent with various embodiments, in response to a triggering event,the image capture devices 302, 304, 306, 308 can be configured toprovide a notification of the object. Consistent with variousembodiments, the notification of the object can be provided in one of anumber of ways. For example, in certain embodiments, the notification ofthe object may be provided in the form of a voice message delivered viaa mobile device, (e.g., smart phone, smart watch, tablet, etc.) orwearable computing device of the user. In certain embodiments, themobile device of the user may vibrate or make a noise to indicate thatan object meeting the obstruction criteria has been identified in one ormore areas of the home environment 300. In certain embodiments, thenotification may be a voice message that is announced in one or moreareas of the home environment 300 via a speaker, intercom, stereo,television, radio, or other device. In certain embodiments, thenotification of the object meeting the obstruction criteria may beprovided as voice message announced through a dog collar equipped with aspeaker and communicatively connected to the image capturing system.Other methods of providing the notification of the object meeting theobstruction criteria are also possible.

Consistent with various embodiments, the triggering event can be one ofa number of different occurrences, triggers, or stimuli that can bedetected by the sensors or other method. In certain embodiments, thetriggering event may be a home arrival (e.g., of a user). The homearrival may be detected in one of a number of ways. In certainembodiments, the home arrival may be detected by a security system ofthe home environment 300 in response to a door unlock, or the opening ofa garage door. For example, when a user arrives home, he or she mayinsert a key into a door of the home environment 300. Accordingly, thisaction may be detected by a security system of the home environment 300,and the user may be provided with a notification of the object meetingthe obstruction criteria.

Aspects of the present disclosure, in certain embodiments, relate to therecognition that visually impaired users may keep a dog or other pettrained to assist them. Accordingly, in certain embodiments, the homearrival of the user may be detected based on detection of a dog barking.In certain embodiments, the home environment may be equipped with asound capture device, such as a microphone, configured to record a setof dog barks. In response to recording the set of dog barks, the soundcapture device or other system may analyze the bark frequency and thebark pitch of the set of dog barks. The bark frequency and the barkpitch may vary based on the individual who has arrived at the home. Forinstance, for an unknown individual, the bark pitch may generally belower (e.g., to intimidate an unknown individual), while, for arecognized individual, the bark pitch may be higher. Additionally, barkfrequency may be greater for an unknown individual than for a knownindividual. In certain embodiments, the barking habits of a particulardog in response to known individuals and unknown individuals may berecorded and analyzed to create a baseline bark profile. Accordingly,the bark pitch and bark frequency of the set of dog barks can becompared to the baseline bark profile to indicate whether it is likelythat the home arrival is a known user or an unknown user. In response todetermining that the bark pitch and bark frequency indicate a homearrival of a known user, the notification of the object meeting theobstruction criteria may be provided to the user. Although bark pitchand bark frequency are specifically referenced herein, other barkfeatures are also contemplated.

In certain embodiments, the triggering event may be an area transition.The area transition may, in certain embodiments, a user moving from onearea of the home environment 300 to another area of the home environment300. For example, the area transition may include a user moving from thedining room to the living room. Aspects of the present disclosure, incertain embodiments, relate to the recognition that visually impairedusers may make use of a cane, walker, or other assistive device tonavigate the areas of the home environment 300. Accordingly, in certainembodiments, the area transition may be detected by a radio-frequencyidentification (RFID) tracking system configured to monitor the locationof a cane (or other assistive device) of the user. In certainembodiments, the cane of the user may be equipped with an RFID bitcommunicatively connected to the RFID tracking system. In certainembodiments, the RFID tracking system can be configured to detect thatthe location of the cane of the user has been displaced by a distancevalue greater than a distance threshold. Consider the following example.In certain embodiments, the distance threshold may be set to 3 meters. Auser may walk, using his or her cane, from the master bedroom to thefamily room of the home environment 300, traveling a distance of 7meters. The RFID tracking system may detect that the location of thecane has moved by a distance of 7 meters, greater than the distancethreshold of 3 meters. Accordingly, notification of the object meetingthe obstruction criteria may be provided to the user.

In certain embodiments, the area transition may be detected by a thermalimaging system configured to monitor a body heat signature. The thermalimaging system may include one or more thermal cameras locatedthroughout the home environment 300. In certain embodiments, the bodyheat signature may be that of a visually impaired user. In certainembodiments, the body heat signature may be that of a dog or other petof the visually impaired user. The thermal imaging system may beconfigured to detect that the body heat signature has been displaced bya distance value greater than a distance threshold. For instance, incertain embodiments, the distance threshold may be set to 1.5 meters. Auser may walk from the kitchen to the living room of the homeenvironment 300, traveling a distance of 3 meters. The thermal imagingsystem may detect that the body heat signature of the user has moved bya distance of 3 meters, greater than the distance threshold of 1.5meters. Accordingly, notification of the object meeting the obstructioncriteria may be provided to the user.

Consider the following example. A visually impaired user may be relaxingby watching television in the living room of the home environment 300. Afaithful guide dog named Toby may be resting comfortably nearby thevisually impaired user. While the user is watching TV, a picture framehanging on the wall in the hallway may fall because the fastener holdingit to the wall broke. Due to the noise of the television, the user maynot have heard the sound of the picture frame falling. In certainembodiments, the image capturing device 304 may capture an image of thehallway (e.g., a second set of images) and compare the image with abaseline set of images taken at a previous time. The image capturingdevice 304 may use a digital image processing algorithm and objectrecognition techniques to identify that the picture frame was not lyingon the floor in the baseline set of images. The image capturing device304 may further determine that the picture frame measures 80 centimetersby 100 centimeters by 5 centimeters, larger than an object sizethreshold of 15 centimeters by 15 centimeters by 15 centimeters. At sometime later, the user may rise to go use the bathroom, and begin walkingout of the living room. The distance threshold may be 2.5 meters. Whenthe user surpasses the distance threshold of 2.5 meters from hisoriginal position, the image capturing system may send a messagenotifying the user of the obstacle in the hallway. The message may beprovided as a voice message played through a speaker in the collar ofthe faithful guide dog named Toby.

FIG. 4 is an illustration of a home environment 400 configured for soundanalysis, consistent with embodiments of the present disclosure. Aspectsof FIG. 4 are directed toward using one or more sound capture devices402, 404, 406, 408, 410 to collect environment data about one or moreareas and identify a change in the home environment for a visuallyimpaired user. As described herein, in certain embodiments, the homeenvironment 400 may be a house, apartment, townhome, condominium, orother residence. The home environment 400 may include one or more areas,including a master bedroom, additional bedrooms, a family room, diningroom, living room, kitchen, and multiple bathrooms. Other areas are alsopossible. As shown in FIG. 4, the sound capture devices 402, 404, 406,408, 410 may be located throughout the home environment 400. Forexample, in certain embodiments, the sound capture device 402 may belocated in the master bedroom, the sound capture device 404 may belocated in a bedroom, the sound capture device 406 may be located in thefamily room, the sound capture device 408 may be located in the kitchen,and the sound capture device 410 may be located in the living room.Consistent with embodiments of the present disclosure, the sound capturedevices 402, 404, 406, 408, 410 may be household appliances configuredfor sound capture, such as televisions, radios, webcams, securitysystems, stereos, computers, mobile devices, and the like.

Aspects of the present disclosure, in certain embodiments, are directedtoward a home environment 400 equipped with one or more sound capturedevices 402, 404, 406, 408, 410 configured to collect environmental dataabout one or more areas of the home environment, and identify a changein the home environment for a visually impaired user. The sound capturedevices 402, 404, 406, 408, 410 may be one or a combination of multipledifferent implementations. For example, the sound capture devices 402,404, 406, 408, 410 may include microphones capable of receiving andrecording audio data such as sound waves. The audio data captured by thesound capture devices 402, 404, 406, 408, 410 may be maintained on localstorage devices or on remote storage devices (e.g., the cloud)communicatively connected with the sound capture devices 402, 404, 406,408, 410.

Consistent with various embodiments of the present disclosure, the soundcapture devices 402, 404, 406, 408, 410 may be configured to collectenvironment data in the form of a first sonar map of the areas of thehome environment 400. The first sonar map may serve as a “baseline” ofthe home environment 400,

Consistent with various embodiments, the sound capture devices 402, 404,406, 408, 410 may be configured to collect environment data including afirst sonar map of one or more areas of the home environment 400. Thefirst sonar map may serve as a “baseline” of the home environment 400,or a reference for use in identifying a change in the home environment400. The first sonar map may capture architectural features (e.g., thelocation of walls, doors) as well as objects (e.g., furniture,appliances). As an example, the image capture devices sound capturedevices 402, 404, 406, 408, 410 may capture a first sonar map of themaster bedroom, family room, dining room, and living room. In certainembodiments, the sound capture devices 402, 404, 406, 408, 410 may becommunicatively connected to the motion capturing devices 202, 204, 206described in FIG. 2. Accordingly, in certain embodiments, in response toreceiving a motion alert from one or more of the motion capturingdevices 202, 204, 206 of FIG. 2, the sound capture devices 402, 404,406, 408, 410 may be configured to capture a second sonar map for theset of areas of the home environment 400. The motion alert from themotion capturing devices 202, 204, 206 may be a notification of motiondetected in one or more areas of the home environment 400. The secondsonar map may be configured to capture the same areas of the homeenvironment 400 as captured by the first sonar map.

The first sonar map and the second sonar map may be captured in one of anumber of ways. In certain embodiments, capturing the first and secondsonar map can include emitting a sound probe by the sound capturedevices 402, 404, 406, 408, 410. The sound capture devices 402, 404,406, 408, 410 can then listen for sound waves reflected from thefurniture, walls, and other features of the home environment 400 inresponse to the sound probe, and use the reflected sound waves to createa three-dimensional sonar map of the features of the home environment400. In certain embodiments, multiple sound probes may be emitted by thesound capture devices 402, 404, 406, 408, 410. In certain embodiments,the sound capture devices 402, 404, 406, 408, 410 may be configured tocollect a background noise level of the home environment 400. Thebackground noise level may include noises such as conversation, thesound of a television, radio, or computer, fan, or the like. In certainembodiments, the background noise level may be distinguished from thesound waves received in response to the sound probe based on frequencyand wavelength differences. Accordingly, the background noise level maynot be used in creating the first and second sonar map.

Consistent with various embodiments of the present disclosure, the soundcapture devices 402, 404, 406, 408, 410 can be configured to compare thefirst sonar map and the second sonar map to one another to identify anenvironmental change in the home environment 400 (e.g., an objectpresent in the second sonar map that was not present in the first sonarmap). In certain embodiments, the sound capture devices 402, 404, 406,408, 410 can use digital image processing techniques similar to thoseused by the image capture devices 302, 304, 306, 308 of FIG. 3. Incertain embodiments, the sound capture devices 402, 404, 406, 408, 410may be configured to identify an object meeting an obstructionthreshold. As described herein, the object threshold may include anobject displacement threshold based on an origin position and asubsequent position of the object, and an object size threshold based ona measurement of the length, width, and height of the object.

In certain embodiments, in response to a triggering event, the soundcapture devices 402, 404, 406, 408, 410 can be configured to provide anotification of the object meeting the obstruction criteria. In certainembodiments, the sound capture devices 402, 404, 406, 408, 410 can beconfigured to provide the notification using techniques and methodssimilar to those used by the image capture devices 302, 304, 306, 308 ofFIG. 3. For example, the notification of the object may be provided tothe visually impaired user as a voice message announced through a mobiledevice, or over an intercom or speaker system of the home environment400. In certain embodiments, the notification of the object may beprovided through a particular household device that the user is in closeproximity with (e.g., through the speakers of a television that the useris watching.) The proximity of a user to a particular household devicemay be determined by an RFID tracking system or by a thermal imagingsystem, as described herein.

In certain embodiments, the triggering event may be a signal or an alertfrom a sleep sensor configured to monitor the sleep cycle of a user. Asan example, in certain embodiments, the triggering event may includedetecting, by the sleep sensor, that a user has awakened from sleeping.The sleep sensor may be configured to monitor one or more biologicalsigns of the user, such as heart rate, respiration frequency, neuralbehavior, eye movement, snoring, and the like. For example, the sleepsensor may determine that a user has awoken based on a pulse (70 beatsper minute) that has increased beyond a pulse threshold (65 beats perminute), or a respiration frequency (14 breaths per minute) that hasincreased beyond a respiration threshold (12 breaths per minute). Thepulse threshold and the respiration threshold may be determined by thesleep sensor based on typical sleeping heart rate and respiration ratedata of the user. Consider the following example. A user may be sleepingin a bedroom of the home environment 400. While the user is sleeping,another inhabitant of the home environment may move a chair in thekitchen to reach a cookie jar on top of the refrigerator before retiringto his or her room, without replacing the kitchen chair. The soundcapture devices 402, 404, 406, 408, 410 may detect the new position ofthe chair by comparing a first sonar map to a second sonar map, anddetermining that the chair meets an obstruction criteria (e.g., adisplacement of more than 50 centimeters from its origin position).Accordingly, in response to determining that the user has awoken, thesound capture devices 402, 404, 406, 408, 410 can provide the user witha notification of the obstruction.

FIG. 5 depicts a high-level block diagram of a computer system 500 forimplementing various embodiments. The mechanisms and apparatus of thevarious embodiments disclosed herein apply equally to any appropriatecomputing system. The major components of the computer system 500include one or more processors 502, a memory 504, a terminal interface512, a storage interface 514, an I/O (Input/Output) device interface516, and a network interface 518, all of which are communicativelycoupled, directly or indirectly, for inter-component communication via amemory bus 506, an I/O bus 508, bus interface unit 509, and an I/O businterface unit 510.

The computer system 500 may contain one or more general-purposeprogrammable central processing units (CPUs) 502A and 502B, hereingenerically referred to as the processor 502. In embodiments, thecomputer system 500 may contain multiple processors; however, in certainembodiments, the computer system 500 may alternatively be a single CPUsystem. Each processor 502 executes instructions stored in the memory504 and may include one or more levels of on-board cache.

In embodiments, the memory 504 may include a random-access semiconductormemory, storage device, or storage medium (either volatile ornon-volatile) for storing or encoding data and programs. In certainembodiments, the memory 504 represents the entire virtual memory of thecomputer system 500, and may also include the virtual memory of othercomputer systems coupled to the computer system 500 or connected via anetwork. The memory 504 can be conceptually viewed as a singlemonolithic entity, but in other embodiments the memory 504 is a morecomplex arrangement, such as a hierarchy of caches and other memorydevices. For example, memory may exist in multiple levels of caches, andthese caches may be further divided by function, so that one cache holdsinstructions while another holds non-instruction data, which is used bythe processor or processors. Memory may be further distributed andassociated with different CPUs or sets of CPUs, as is known in any ofvarious so-called non-uniform memory access (NUMA) computerarchitectures.

The memory 504 may store all or a portion of the various programs,modules and data structures for processing data transfers as discussedherein. For instance, the memory 504 can store a home environmentmonitoring application 550. In embodiments, home environment monitoringapplication 550 may include instructions or statements that execute onthe processor 502 or instructions or statements that are interpreted byinstructions or statements that execute on the processor 502 to carryout the functions as further described below. In certain embodiments,the home environment monitoring application 550 is implemented inhardware via semiconductor devices, chips, logical gates, circuits,circuit cards, and/or other physical hardware devices in lieu of, or inaddition to, a processor-based system. In embodiments, the homeenvironment monitoring application 550 may include data in addition toinstructions or statements.

The computer system 500 may include a bus interface unit 509 to handlecommunications among the processor 502, the memory 504, a display system524, and the I/O bus interface unit 510. The I/O bus interface unit 510may be coupled with the I/O bus 508 for transferring data to and fromthe various I/O units. The I/O bus interface unit 510 communicates withmultiple I/O interface units 512, 514, 516, and 518, which are alsoknown as I/O processors (IOPs) or I/O adapters (IOAs), through the I/Obus 508. The display system 524 may include a display controller, adisplay memory, or both. The display controller may provide video,audio, or both types of data to a display device 526. The display memorymay be a dedicated memory for buffering video data. The display system524 may be coupled with a display device 526, such as a standalonedisplay screen, computer monitor, television, or a tablet or handhelddevice display. In one embodiment, the display device 526 may includeone or more speakers for rendering audio. Alternatively, one or morespeakers for rendering audio may be coupled with an I/O interface unit.In alternate embodiments, one or more of the functions provided by thedisplay system 524 may be on board an integrated circuit that alsoincludes the processor 502. In addition, one or more of the functionsprovided by the bus interface unit 509 may be on board an integratedcircuit that also includes the processor 502.

The I/O interface units support communication with a variety of storageand I/O devices. For example, the terminal interface unit 512 supportsthe attachment of one or more user I/O devices 520, which may includeuser output devices (such as a video display device, speaker, and/ortelevision set) and user input devices (such as a keyboard, mouse,keypad, touchpad, trackball, buttons, light pen, or other pointingdevice). A user may manipulate the user input devices using a userinterface, in order to provide input data and commands to the user I/Odevice 520 and the computer system 500, and may receive output data viathe user output devices. For example, a user interface may be presentedvia the user I/O device 520, such as displayed on a display device,played via a speaker, or printed via a printer.

The storage interface 514 supports the attachment of one or more diskdrives or direct access storage devices 522 (which are typicallyrotating magnetic disk drive storage devices, although they couldalternatively be other storage devices, including arrays of disk drivesconfigured to appear as a single large storage device to a hostcomputer, or solid-state drives, such as flash memory). In someembodiments, the storage device 522 may be implemented via any type ofsecondary storage device. The contents of the memory 504, or any portionthereof, may be stored to and retrieved from the storage device 522 asneeded. The I/O device interface 516 provides an interface to any ofvarious other I/O devices or devices of other types, such as printers orfax machines. The network interface 518 provides one or morecommunication paths from the computer system 500 to other digitaldevices and computer systems; these communication paths may include,e.g., one or more networks 530.

Although the computer system 500 shown in FIG. 5 illustrates aparticular bus structure providing a direct communication path among theprocessors 502, the memory 504, the bus interface 509, the displaysystem 524, and the I/O bus interface unit 510, in alternativeembodiments the computer system 500 may include different buses orcommunication paths, which may be arranged in any of various forms, suchas point-to-point links in hierarchical, star or web configurations,multiple hierarchical buses, parallel and redundant paths, or any otherappropriate type of configuration. Furthermore, while the I/O businterface unit 510 and the I/O bus 508 are shown as single respectiveunits, the computer system 500 may, in fact, contain multiple I/O businterface units 510 and/or multiple I/O buses 508. While multiple I/Ointerface units are shown, which separate the I/O bus 508 from variouscommunications paths running to the various I/O devices, in otherembodiments, some or all of the I/O devices are connected directly toone or more system I/O buses.

In various embodiments, the computer system 500 is a multi-usermainframe computer system, a single-user system, or a server computer orsimilar device that has little or no direct user interface, but receivesrequests from other computer systems (clients). In other embodiments,the computer system 500 may be implemented as a desktop computer,portable computer, laptop or notebook computer, tablet computer, pocketcomputer, telephone, smart phone, or any other suitable type ofelectronic device.

FIG. 5 depicts several major components of the computer system 500.Individual components, however, may have greater complexity thanrepresented in FIG. 5, components other than or in addition to thoseshown in FIG. 5 may be present, and the number, type, and configurationof such components may vary. Several particular examples of additionalcomplexity or additional variations are disclosed herein; these are byway of example only and are not necessarily the only such variations.The various program components illustrated in FIG. 5 may be implemented,in various embodiments, in a number of different manners, includingusing various computer applications, routines, components, programs,objects, modules, data structures, etc., which may be referred to hereinas “software,” “computer programs,” or simply “programs.”

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present disclosurehave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A computer implemented method for detecting achange in a home environment for a visually impaired user, the methodcomprising: mapping, using a sonar mapping system, a first sonar map ofa set of areas of the home environment; mapping, in response toreceiving a noise alert from a sound detection system coupled to thesonar mapping system, a second sonar map of the set of areas;identifying an object of the second sonar map meeting an obstructioncriterion; and providing a notification to the visually impaired userthat the identified object meets the obstruction criterion, in responseto an alert from a sleep sensor.
 2. The method of claim 1, wherein thealert from the sleep sensor includes a change in a pulse beyond a pulsethreshold, and a change in respiration frequency beyond a respirationfrequency threshold.
 3. The method of claim 1, further comprising:collecting, by the sonar mapping system, a background noise level of theset of areas; collecting, by the sonar mapping system, a sound baselineof the set of areas; and creating, using the sound baseline offset bythe background noise level, the sonar map.
 4. A system for detecting achange in a home environment for a visually impaired user, the systemcomprising: a plurality of sound capture devices each configured to emitone or more sound probes in the home environment and to detect reflectedsound waves; a sleep sensor configured to monitor one or more biologicalsigns of the visually impaired user; and a processor communicativelycoupled to the plurality of sound capture devices and the sleep sensor;wherein the processor is configured to generate a first sonar map of aset of areas of the home environment based on the reflected sound wavesdetected by the plurality of sound capture devices, and to generate asecond sonar map based on subsequent reflected sound waves in responseto receiving a noise alert; wherein the processor is further configuredto compare the second sonar map to the first sonar map to identify anobject of the second sonar map meeting an obstruction criterion; and twherein the processor is further configured to provide a notification tothe visually impaired user that the identified object meets theobstruction criterion in response to an alert from the sleep sensorindicating that the visually impaired user has awoken.
 5. The system ofclaim 4, wherein the alert from the sleep sensor includes a change in apulse beyond a pulse threshold, and a change in respiration frequencybeyond a respiration frequency threshold.
 6. The system of claim 4,wherein the plurality of sound capture devices are further configured tocollect a background noise level of the set of areas and a soundbaseline of the set of areas; and wherein the processor is configured togenerate the first and second sonar maps using the sound baseline offsetby the background noise level.
 7. A non-transitory computer programproduct comprising a computer readable storage medium having a computerreadable program stored therein, wherein the computer readable program,when executed on a first computing device, causes the first computingdevice to: map a first sonar map of the set of areas of the homeenvironment; map, in response to receiving a noise alert from a sounddetection system, a second sonar map of the set of areas; identify anobject of the second sonar map meeting an obstruction criterion; andproviding a notification to the visually impaired user that theidentified object meets the obstruction criterion, in response to analert from a sleep sensor.
 8. The non-transitory computer programproduct of claim 7, wherein the alert from the sleep sensor includes achange in a pulse beyond a pulse threshold, and a change in respirationfrequency beyond a respiration frequency threshold.
 9. Thenon-transitory computer program product of claim 7, wherein the computerreadable program, when executed on the first computing device, furthercauses the first computing device to generate the first and second sonarmaps using a sound baseline offset by a background noise level.